SEARCHING FOR VIRTUAL REALITY Virtual Reality is a way for humans to visualize, manipulate and interact with computers and extremely complex data. For some, VR is a specific collection of technologies like a head mounted display, and glove input device and audio while others include conventional books, movies or pure fantasy, imagination, and visualization. The visualization part refers to the computer generating visual, auditory or other sensual outputs to the user of a world within the computer. This may be a CAD model, a scientific simulation, or a view into a database. The user can interact with the world and directly manipulate objects within the world. Some worlds are animated by other processes like physical simulations, or simple animation scripts. Interaction with the virtual world, at least with near real time control of the viewpoint, is a critical test for a virtual reality. Some people object to the term "Virtual Reality", saying it is an oxymoron. Other terms that have been used are Synthetic Environments, Cyberspace, Artificial Reality, Simulator Technology, etc. VR is the most common. The applications being developed for VR run a wide spectrum, from games to architectural and business planning. Many applications are worlds that are very similar to our own, like CAD (Computer assisted drafting) or architectural modeling. Some applications provide ways of viewing from an advantageous perspective not possible with the real world, like scientific simulators and telepresense systems, air traffic control systems. Other applications are much different from anything we have ever directly experienced before. These may be the hardest, and most interesting systems. Take for instance, visualizing the ebb and flow of the world's financial markets, or navigating a large corporate information base. A major distinction of VR systems is the mode with which they interface to the user. Some systems use a conventional computer monitor to display the visual world. These are sometimes called Desktop VR or a Window on a World (WoW). This concept traces its lineage back through the entire history of computer graphics. In 1965, Ivan Sutherland laid out a research program for computer graphics in a paper called "The Ultimate Display" that has driven the field for the past nearly thirty years. "One must look at a display screen," Sutherland said, "as a window through which one beholds a virtual world. The challenge to computer graphics is to make the picture in the window look real, sound real and the objects act real." A variation of the WoW approach merges a video input of the user's silhouette with a 2D computer graphic. The user watches a monitor that shows his body's interaction with the world. Myron Kruger championed this form of VR since the late 60's. He published two books on the subject: "Artificial Reality" and "Artificial Reality II". At least one commercial system uses this approach, the Mandala system. A version of the Mandala is used by the cable TV channel Nickelodeon for a game show (Nick Arcade) to put the contestants into what appears to be a large video game. The ultimate VR systems completely immerse the user's personal viewpoint inside the virtual world. These "immersive" VR systems are often equipped with a Head Mounted Display (HMD). This is a helmet or a face mask that holds the visual and auditory displays. The helmet may be free ranging, tethered, or it might be attached to some sort of a boom armature. A nice variation of the immersive systems use multiple large projection displays to create a 'Cave' or room in which the viewer(s) stand. An early implementation was called "The Closet Cathedral" for the ability to create the impression of an immense environment within a small physical space. The Holodeck used in the television series "Star Trek: The Next Generation" is an extrapolation of this technology. Many of the displays at the Universal Studios in California and Florida are this type of VR. Telepresence is a variation on visualizing complete computer generated worlds. This technology links remote sensors in the real world with the senses of a human operator. Fire fighters use remotely operated vehicles to handle some dangerous conditions. Surgeons are using very small instruments on cables to do surgery without cutting a major hole in their patients. The instruments have a small video camera at the business end. Robots equipped with telepresence systems have already changed the way deep sea and volcanic exploration is done. NASA plans to use telerobotics for space exploration. There is currently a joint US/Russian project researching telepresence for space rover exploration. Merging the Telepresence and Virtual Reality systems gives the Mixed Reality or Seamless Simulation systems. Here the computer generated inputs are merged with telepresence inputs and/or the users view of the real world. A surgeon's view of a brain surgery is overlaid with images from earlier CAT scans and real-time ultrasound. A fighter pilot sees computer generated maps and data displays inside his fancy helmet visor or on cockpit displays. There are a number of specialized types of hardware devices used for Virtual Reality applications. One of the most time consuming tasks in a VR system is the generation of the images. Fast computer graphics opens a very large range of applications aside from VR. One key element for interaction with a virtual world, is a means of tracking the position of a real world object, such as a head or hand. There are numerous methods for position tracking and control. Ideally a technology should provide 3 measures for position (X, Y, Z) and 3 measures of orientation (roll, pitch, yaw). The simplest control hardware is a conventional mouse, trackball or joystick. While these are two dimensional devices, creative programming can use them for 6D controls. There are a number of 3 and 6 dimensional mice/trackball/joystick devices being introduced to the market at this time. These add some extra buttons and wheels that are used to control not just the XY translation of a cursor, but its Z dimension and rotations in all three directions. The Global Devices 6D Controller is one such 6D joystick. It looks like a racket ball mounted on a short stick. You can pull and twist the ball in addition to the left/right & forward/back of a normal joystick. One common VR device is the instrumented glove. The use of a glove to manipulate objects in a computer is covered by a basic patent in the USA. Such a glove is outfitted with sensors on the fingers as well as an overall position/orientation tracker. There are a number of different types of sensors that can be used. VPL (holders of the patent) made several DataGloves, mostly using fiber optic sensors for finger bends and magnetic trackers for overall position. Mattel manufactured the PowerGlove for use with the Nintendo game system, for a short time. This device is easily adapted to interface to a personal computer. It provides some limited hand location and finger position data using strain gauges for finger bends and ultrasonic position sensors. Stereo vision is often included in a VR system. This is accomplished by creating two different images of the world, one for each eye. The images are computed with the viewpoints offset by the equivalent distance between the eyes. There are a large number of technologies for presenting these two images. The images can be placed side-by-side and the viewer asked (or assisted) to cross their eyes. The images can be projected through differently polarized filters, with corresponding filters placed in front of the eyes. One hardware device closely associated with VR is the Head Mounted Device (HMD). These use some sort of helmet or goggles to place small video displays in front of each eye, with special optics to focus and stretch the perceived field of view. Most HMDs use two displays and can provide stereoscopic imaging. Others use a single larger display to provide higher resolution, but without the stereoscopic vision. The 'Entry Level' VR system takes a stock personal computer or work station and implements a WoW system. The system may be based on an IBM clone (MS-DOS/Windows) machine or an Apple Macintosh, or perhaps a Commodore Amiga. The DOS type machines (IBM PC clones) are the most prevalent. Whatever the base computer it includes a graphic display, a 2D input device like a mouse, trackball or joystick, the keyboard, hard disk & memory. The next step up from an EVR system adds some basic interaction and display enhancements like a stereographic viewer (LCD Shutter glasses) and a input/control device such as the Mattel PowerGlove and/or a multidimensional (3D or 6D) mouse or joystick. The final step up the VR technology ladder is to add a rendering accelerator and/or frame buffer and possibly other parallel processors for input handling, etc. The simplest enhancement in this area is a faster display card. These can make a dramatic improvement in the rendering performance of a desktop VR system. An AVR system might also add a sound card to provide mono, stereo or true 3D audio output. Some sound cards also provide voice recognition. A common variation on VR is to use a Cockpit or Cab compartment to enclose the user. The virtual world is viewed through some sort of view screen and is usually either projected imagery or a conventional monitor. The cockpit simulation is very well known in aircraft simulators, with a history dating back to the early Link Flight Trainers (1929?). The cockpit is often mounted on a motion platform that can give the illusion of a much larger range of motion. Cabs are also used in driving simulators for ships, trucks, and tanks. One of the biggest VR projects is the Defense Simulation Internet. This project is a standardization being pushed by the USA Defense Department to enable diverse simulators to be interconnected into a vast network. It is an outgrowth of the Defense Advanced Research Projects Administration (DARPA) SIMNET project of the later 1980s. SIMNET is a collection of tank simulators (Cab type) that are networked together to allow unit tactical training. Simulators in Germany can operate in the same virtual world as simulators in the USA, partaking of the same battle exercise. Just what is required of a VR program? The basic parts of the system can be broken down into an Input Processor, a Simulation Processor, a Rendering Process, and a World Database. All these parts must consider the time required for processing. Every delay in response time degrades the feeling of 'presence' and reality of the simulation. The Input Processes of a VR program control the devices used to input information to the computer. There are a wide variety of possible input devices: keyboard, mouse, trackball, joystick, 3D & 6D position trackers (glove, wand, head tracker, body suit, etc.). A voice recognition system also augments a good VR system, especially if the user's hands are being used for other tasks. Generally, the input processing of a VR system is kept simple. The object is to get the coordinate data to the rest of the system with minimal lag time. Some position sensor systems add some filtering and data smoothing processing. Some glove systems add gesture recognition. This processing step examines the glove inputs and determines when a specific gesture has been made. Thus it can provide a higher level of input to the simulation. The core of a VR program is the simulation system. This is the process that knows about the objects and the various inputs. It handles the interactions, the scripted object actions, simulations of physical laws (real or imaginary) and determines the world status. This simulation is basically a discrete process that is iterated once for each time step or frame. Coordination of these can be a complex task. The simulation engine takes the user inputs along with any tasks programmed into the world such as collision detection, scripts, etc. and determines the actions that will take place in the virtual world. The Rendering Processes of a VR program are those that create the sensations that are output to the user. There would be separate rendering processes for visual, auditory, haptic (touch/force), and other sensory systems. Each renderer takes a description of the world state from the simulation process or derives it directly from the World Database for each time step. The visual renderer is the most common process and it has a long history from the world of computer graphics and animation. The major consideration of a graphic renderer for VR applications is the frame generation rate. It is necessary to create a new frame every 1/20 of a second or faster. Twenty frames per second (fps) is roughly the minimum rate at which the human brain will merge a stream of still images and perceive a smooth animation. This requirement eliminates a number of rendering techniques such as raytracing and radiosity. Haptics is the generation of touch and force feedback information. This area is a very new science and there is much to be learned. There have been very few studies done on the rendering of true touch sense (such as liquid, fur, etc.). Almost all systems to date have focused on force feedback and kinesthetic senses. These systems can provide good clues to the body regarding the touch sense, but are considered distinct from it. Many of the haptic systems thus far have been exo-skeletons that can be used for position sensing as well as providing resistance to movement or active force application. The sense of balance and motion can be served to a fair degree in a VR system by a motion platform. These are used in flight simulators and some theaters to provide some motion cues that the mind integrates with other cues to perceive motion. It is not necessary to recreate the entire motion perfectly to fool the mind into a willing "suspension of disbelief." Other senses such as taste, smell, pheromone, etc. are beyond our ability to render rapidly and effectively. We just don't know enough about the functioning of these other senses. A virtual world consisting only of static objects is only of mild interest. VR enthusiasts state that interaction is the key to a successful and interesting virtual world. The times, they are a' changing. Virtual Reality is being used in nearly every human endeavor. It's only a matter of time before we begin to experience it in our daily lives. Who knows, perhaps someday we'll do the laundry, shop, and perform other daily chores in virtual, and not real-ality...